The self-evaluation document

The self-evaluation will be required approximately one month before commencement of the period (usually the academic year) in which the subject review will take place. If a review period commences later in the academic year, a submission date will be agreed with the institution.

For reviews taking place in spring 2002 the likely deadline is 1 December 2001 for submission of the self-evaluation. For reviews taking place in 2002/03 the deadline is likely to be 1 September 2002.

(Note though that these dates are subject to agreement between your institution and the QAA and may vary from institution to institution.)

The scale of the task of drafting, re-drafting, editing and polishing the self-evaluation should not be underestimated.

Key points

• ensure you are absolutely clear about the rubric (consult your central quality team if in any doubt)
• share drafts widely:
  • within the department
  • with students
  • with trusted external colleagues
  • with central service providers (for appropriate sections)
• use a single writer/editor, supported by ‘critical friends’ – some consistency in the style and tone of the text is important

Guidelines for producing self-evaluation documents

The self-evaluation document is central to the process of subject review, and fulfils two main functions.

First, it is intended to encourage the subject provider to evaluate the quality of the learning opportunities offered to students and the standards achieved by them. It provides an opportunity for the staff of the subject provider to reflect on ‘what do we do?’, ‘why we do it’, and ‘why do we do it in the way that we do?’.

Subject reviewers will expect to see evidence of careful self-analysis. This should involve an evaluation of the perceived strengths of the provision, with reference to the evidence which justifies the statements made, and of weaknesses, where these are recognised. Where weaknesses are acknowledged, the subject provider is encouraged to discuss the issues and the steps being taken to bring about improvements.

Second, the document provides a framework for a process of subject review based on the testing and verification of statements made by subject providers.

The document should reflect on current provision in a manner that evaluates both strengths and weaknesses, indicates the changes that have taken place since earlier external reviews, and considers what may be necessary to change in the future. It is the most important of the small number of documents made available to reviewers in advance of a review.

Annex C of the QAA handbook describes in detail the structure and use made of the self-evaluation.

QAA has also offered the following comments about the importance of the self-evaluation:

“It is worth re-iterating the crucial importance of the self-evaluation document to the subject review process. This is particularly important because it is already clear that the quality of the self-evaluation documents produced by Scottish institutions for 2000-1 is variable.

Self-evaluation, by concentrating on the strengths of the provision as perceived by the institution and by reviewers in previous subject review(s), and on how weaknesses perceived by the institution and reviewers have been addressed, can have a considerable effect on the intensity of review.

A robust, transparent and well-written self-evaluation will ensure that reviewers pursue only those matters that are central to quality and standards, leading to the reality of a ‘light touch’ review.

A poorly written self-evaluation, or one which obfuscates and attempts to cover up weaknesses, may well cause reviewers to pursue a range of matters over a greater period of time in order to arrive at their judgements. This in turn will lead to greater intensity of review.”

— (QAA. Letter to institutions from Peter Milton, November 2000)

Self-evaluations MUST:

• include clear aims
• follow the structure set out by QAA
• be supported by evidence
• give a clear description and evaluation of courses designed to meet the subject aims and the learning outcomes set out in programme specifications
• discuss both strengths and weaknesses
• cover steps being taken to remedy weaknesses
• be genuinely self-critical

Key points:

• although there is some flexibility on length, the self-evaluation should not exceed the maximum word and page limits if it does, QAA may return the self-evaluation for editing

• aims must be clearly expressed again, the self-evaluation may be returned for re-drafting if QAA has doubts about the clarity of aims

• the self-evaluation guides all preparations for the visit and is the starting point for the review team’s enquiries it is vital therefore to ensure that a high quality document is submitted

• it is extremely important to ensure that the aims are right it is essential that these are accurate, clearly expressed and supported by evidence

Use of the self-evaluation

Remember, subject review teams will:

• test the rigour of the self-evaluation and use this as a guide to whether a lighter touch can be applied

• check the accuracy of the description of the provision
• report on the rigour and openness of the self-evaluation
• expect a clear identification of the range of subject provision (including all taught provision)

Comments on the self-evaluation

Some thoughts on the contents:

• section Bii should evaluate how learning outcomes are being achieved through the curriculum
• in section Bii, think how assessment methods are linked to learning outcomes
• with regard to student achievement, you might consider including a quote from a student or an employer
• what are the rules for progression, are they clear and are students aware of them?
• how do you support students, especially weaker ones, to enable them to achieve the learning outcomes set?
• the self-evaluation should evaluate how well your system for student support and guidance works in the department/school and explain the procedures in place to identify students with academic problems and to follow these up
• questions to address in relation to learning resources: to what extent do resources contribute to delivery of aims? are they fit for purpose? do students use them? do they use them effectively? what are the resource problems? how are you tackling them (where feasible)?
• do the procedures for the maintenance and enhancement of standards and quality WORK? are there problems with the procedures that prevent you from meeting your aims? how do you seek student feedback/opinion and how responsive is the department/school to student opinion?
• identify clearly the department/school’s quality assurance procedures and the forum in which quality issues are discussed
• section Biv signals the extent to which staff are open to new ideas, self-evaluative etc

Some other tips:

• annexes – QAA may be more flexible on word limits for annexes IF it seems to be justified (for example by inclusion of particularly helpful tables/charts), but note that this flexibility has yet to be tested under the new method
• the word limit may be rigorously applied by the QAA. Main headings are not counted although sub-headings and sub-paragraph numbers are.
• programme specifications should, as far as is possible, be completed and to hand as a reference, PRIOR to the drafting of the self-evaluation
• write in clear, plain English, avoiding jargon and cliche
• take some time to ensure the layout of the document is clear and comprehensible

Quantitative data

The quantitative data submitted with the self-evaluation document are often provided by your central academic registry. Experience has shown that difficulties may arise where the subject team does not completely understand or ‘own’ these statistics.

It is vital therefore that these are checked and double-checked by the subject team in discussion with your central staff to ensure that the data matches the department/school’s records, the reason for any variance is known and responses to anticipated queries can be prepared. The data will be referred to by departments/schools in the text of the self-evaluation, for example in relation to student progression and wastage rates.

Note also that where claims are made in aims about providing opportunities for students from a wide range of backgrounds or enabling such students to fulfil their potential, departments/schools must be able to highlight relevant data to evidence the achievement of these.

The structure of the self-evaluation: summary

The Structure of the Self-evaluation: Summary Section	Word Limits
Aims  aims should be succinct but convey clearly the parameters of the subject can be presented as narrative statements, bullet points, or as a mixture of the two	500 words maximum
Evaluation of the subject provision  learning outcomes appropriateness of the intended learning outcomes in relation to the overall aims of the provision, relevant subject benchmark statements, and other external reference points effectiveness of measures to ensure that staff and students have a clear understanding of the aims and intended outcomes of programmes curricula and assessment effectiveness of the content and design of the curricula in enabling the intended outcomes of courses to be achieved effectiveness of student assessment in measuring achievement of the intended outcomes of courses maintenance and enhancement of standards effectiveness of the measures taken to maintain and enhance the quality and standards of provision quantitative data, including statistics on student achievement in all forms of summative assessment, degree classifications, entry qualifications, progression and completion rates, first employment destinations qualitative feedback, including student feedback, external examiners’ reports, employers’ views on graduates they have recruited, accreditation reports by professional or statutory bodies, previous subject reviews, comments from internal annual and periodic review	Total for evaluation section: 6000 words maximum
Annexes  a course specification for each programme covered by the review should be annexed	as required
brief factual explanations may also be provided of:  curricular structures, options and pathways provided in the subject being reviewed	500 words maximum
a brief factual explanation may be provided of:  any relationship with a collaborating institution, for example if a course is being provided jointly, or is franchised	500 words maximum
list of additional documentation available	unspecified

Self-evaluations: essential tips

You should:

• ensure you are analytical, self-critical and evaluative rather than descriptive
• follow the format recommended for aims
• check if every description has an evaluation attached – they must have either immediately afterwards or in a concluding paragraph
• reduce detailed descriptions of provision where possible – bear in mind that details of courses and facilities etc can be provided in depth in supplementary information, which can be sent to subject reviewers during the process of review – also, much in the way of course details will appear in programme specifications
• try to think, in considering the level of detail needed in descriptions, what is the minimum needed to give the reader an understanding of the provision
• ensure that every claim is grounded in evidence and use examples
• use explicit references back from text to your aims (number the aims to make this easier and use fewer words) and to the learning outcomes of courses
• seek to ensure that, insofar as it is possible, you submit subject aims that are in line with your institution’s strategic aims (or at least are not contradictory)
• ensure that statements and claims in the main text match aims and learning outcomes as set out in programme specifications
• ensure that the aims and learning outcomes are reflected in other departmental/school documentation including student handbooks
• identify the strengths of the department/school and offer clear evidence that supports your claims
• be honest about the department/school’s weaknesses and state the strategies you have put in place to deal with them
• show the impact on teaching and learning of particular strengths (for example say HOW high quality research informs teaching)
• try to think of how each area relates to your students’ educational experience – imagine things from the student viewpoint
• be aware that if a weakness has been identified and not yet addressed, there is still time – something introduced just before the subject review visit will be regarded sceptically by reviewers but is still better than nothing
• ensure that claims such as ‘the department/school regularly discusses…’ are evidenced in minutes/notes of meetings
• show that the department is committed to a self-critical approach and to improvement
• consider a single selective quotation from a student feedback form or an external examiner or an employer of your graduates. These can be highly effective in the right place – especially where it provides evidence in support of a claim.
• circulate drafts widely – you might even want to put a version on the Web
• get a group of students to read a draft
• let central service providers see drafts of relevant sections (for example on library or IT provision)
• seek to make the self-evaluation interesting – it should have fizz and a spark about it, reflecting the enthusiasm of the providers for their subject

You shouldn’t:

• forget that the self-evaluation will govern the conduct of your review and may influence whether or not you benefit from a lighter touch
• try to hide problems – better to identify them and show how they are being tackled (indicates self-evaluative subject team too). Reviewers will want to be assured you are dealing with the key issues; perhaps think of issues from students’ viewpoint – what might you appear not to be delivering and for what reasons?

• be untruthful – inconsistencies, weaknesses or dishonesty will be spotted
• use ‘we are proud’ type of statements or over-hype yourselves
• include ‘hostages to fortune’
• forget to state the obvious
• feel that each area of provision has to receive equal coverage
• underestimate the time required to draft the document

Last Modified: 4 June 2010